
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. 



CONFIRMATION NO. 



09/896,199 



24737 



06/29/2001 



Eric Cohen-solal 



7590 



11/05/2003 



PHILIPS INTELLECTUAL PROPERTY & STANDARDS 
P.O. BOX 3001 

BRIARCLIFF MANOR, NY 10510 



US010324 



7568 



EXAMINER 



BELL, PAUL A 



ART UNIT 



PAPER NUMBER 



2675 

DATE MAILED: 1 1/05/2003 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 



Application No. 

09/896.199 



Examiner 

PAUL A BELL 



Applicant(s) 

COHEN-SOLAL. ERIC 



Art Unit 

2675 



The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 21 August 2003 . 
2a)^ This action is FINAL. 2b)n This action is non-finaL 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) IE1 Claim{s) 1-20 is/are rejected. 

Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on is/are: a)n accepted or bD objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

1 1) n The proposed drawing correction filed on is: a)^ approved b)^ disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

1 3) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a)nAII b)n Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2.n Certified copies of the priority documents have been received in Application No. . 



3.n Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 



1) □ Notice of References Cited (PTO-892) 

2) n Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) n tnfonmation Disclosure Statement(s) (PTO-1449) Paper No(s). 



4) O Interview Sumnriary (PTO-41 3) Paper No(s). 

5) D Notice of Informal Patent Application (PTO-152) 

6) n Other: 
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DETAILED ACTION 



Claim Rejections - 35 USC § 103 



1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be patented 
and the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Inagaki (5,999,214) in view of Pavlovic et al. "Integration of audio/visual information 
for use in human-computer intelhgent interaction". Image processing, 1997 Proceedings 
IEEE pages 121. 

With regard to claim 1 Inagaki teaches a video display device comprising: a 
display configured to display a primary image and a picture-in-picture image (PEP) 
overlaying the primary image (figure 11, items 13 and 17); and a processor operatively 
coupled to the display and configured to receive a first video data stream for the primary 
image, to receive a second video data stream for the PIP (figure 11, items 22 and 16), 

Inagaki does not teach, "and to change a PIP display characteristic in response to 
a received audio command and a related gesture from a user". Inagaki instead only uses 
the audio indication from a user and does not depend on a "related gesture from a user" 



(figure 1 1 "VOICE DIRECTION DETECTION UNIT", column 3, lines 31-33, column 
10, lines 16-25). 
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However, Pavlovic et al. does teach integrating both audio command and visual 
gesture information to manipulate virtual objects displayed on the projection screen ( 
page 123 3. EXPERIMENTAL RESULTS section ). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to use a "received audio command and a related gesture from a 
user" as taught by Pavlovic in the apparatus of Inagaki, because of the motivation 
provided by Pavlovic "Psychological studies, for example, show that people prefer to 
use hand gestures in combination with speech in a virtual environment , since they allow 
the user to interact without special training or special apparatus". 

With regard to claim 2 the combination of Inagaki and Pavlovic teaches the 
video display device of claim 1, wherein the PIP display characteristic is at least one of a 
position of the PIP on the display and a display size of the PIP (See Inagaki figure 8a). 

With regard to claim 3 the combination of Inagaki and Pavlovic teaches the 
video display device of claim 1, comprising: a microphone for receiving the audio 
indication from the user; and a camera for acquiring an image of the user containing the 
related gesture (See Inagaki figure 11). 

With regard to claim 4 the combination of Inagaki and Pavlovic teaches the 
video display device of claim 1 wherein the processor is configured to analyze audio 
information received from the user to identify when a PIP related audio indication is 
intended by the user (See Inagaki figure 8a and 8b). 
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With regard to claim 5 the combination of Inagaki and Pavlovic teaches the 
video display device of claim 1, wherein the processor is configured to analyze image 
information received from the user after the audio indication is received to identify the 
change in the PIP display characteristic that is expressed by the received gesture (See 
haagaki figure 8a and 8b and Pavlovic et al figures 6-8). 

With regard to claim 6 the combination of Inagaki and Pavlovic teaches the 
video display device of claim 5, v^herein the image information is contained in a 
sequence of images and wherein the processor is configured to analyze the sequence of 
images to determine the received gesture (since a gesture can be a motion which would 
require a sequence of images to detect this feature is inherent to the system of hiagaki 
and Pavlovic). 

With regard to claim 7 the combination of Inagaki and Pavlovic teaches the 
video display device of claim 1, wherein the image information is contained in a 
sequence of images and wherein the processor is configured to determine the received 
gesture by analyzing the sequence of images and determining a trajectory of a hand of 
the user (since a gesture can be a motion which would require a sequence of images to 
detect this feature is inherent to the system of Inagaki and Pavlovic). 

With regard to claim 8 the combination of Inagaki and Pavlovic teaches the 
video display device of claim 1, wherein the processor is configured to determine the 
received gesture by analyzing an image of the user and determining a posture of a hand 
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of the user (since a gesture can be a posture of a hand this feature is inherent to the 
system of Inagaki and Pavlovic). 

With regard to claim 9 the combination of Liagaki and Pavlovic suggest the 
video display device of claim 1, wherein the video display device is a television (since 
Pavlovic shows a projection screen in figure 6 and since it is also well-known in the 
prior art that televisions use projection screens one would be motivated to have a 
projection screen with a dual use such as conference and watching the game . 

With regard to claim 9 the combination of Liagaki and Pavlovic teaches the 
video display device of claim 1, wherein the image is a sequence of images of the user 
containing the user gesture, the video display device comprising a camera for acquiring 
the sequence of images of the user (see Liagaki figure 11, item 2). 

With regard to claims 1 1-14 the combination of Inagaki and Pavlovic was shown 
with regards to apparatus claims 1-10 to be obvious and therefore the method claims 11- 
14 which corresponds to the apparatus is also obvious. 

With regard to claims 15-18 the combination of Inagaki and Pavlovic was shown 
above to read on most of these limitation in claims 1-10 in addition to summarize a 
feature directed towards a program stored implementing this process is inherent to the 
automatic computer system taught by the combination of Inagaki and Pavlovic. 

With regard to claim 19 the combination of Inagaki and Pavlovic was shown 
above to read on most of these limitation in claims 1-18 in addition to summarize a 
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feature directed towards, "wherein the PIP display characteristic is at least one of a 
position of the PIP on the display and a display size of the PIP" this feature is inherent to 
a system that displays a PIP. 

With regard to claim 20 the combination of Inagaki and Pavlovic was shown 
above to read on most of these limitation in claims 1-19 in addition to summarize a 
feature directed towards , "wherein the processor is configured to analyze image 
information received fi-om the user after the audio indication is received to identify the 
change in the PIP display characteristic that is expressed by the received gesture" (See 
Pavlovic figure 5). 

Response to Arguments 

3. Applicant's arguments filed 8/21/2003 have been fully considered but they are 
not persuasive. 

With regard to claim 1, the applicant argues "Inagaki does not disclose or suggest 
" a processor ...configured... to change a PIP display characteristic in response to a 
received audio command and a related gesture fi-om a user." 

The examiner disagrees because since the Inagaki system responds to any sound 
with the same command it reads on this broad language. Also examiner gave motivation 
for improving the Inagaki system with Pavlovic reference above. 

With regard to claim 1 1, the applicant argues Inagaki does not disclose or 
suggest "determining whether the received audio indications is one of a plurality of 
expected audio indications" and Pavlovic does not teach or suggest "analyzing a gesture 




Application/Control Number: 09/896, 1 99 Page 7 

Art Unit: 2675 

of the user if the received audio indication is one of the pluraUty of expected audio 
indications" 

The examiner disagrees because since the Inagaki system responds to any sound 
(audio indications) with the same command it still reads on this broad language. Also 
examiner gave motivation for improving the Inagaki system with Pavlovic reference 
above. 

Conclusion 



4. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of 
time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated fi-om the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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5. 



Any inquiry concerning this communication or earlier communications from the 



examiner should be directed to Paul Bell whose telephone number is (703) 306-3019. If 
attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Steven Saras, can be reached at (703) 305-9720. 

Any response to this action should be mailed to: 



or faxed to: (703) 872-9314 

Hand-delivered responses should be brought to Crystal Park U, 2121 Crystal Drive, 
Arlington, VA, Sixth Floor (Receptionist). Any inquiry of a general nature or relating to 
the status of this application or proceeding should be directed to the Technology Center 
2600 Customer Service Office whose telephone number is (703) 306-0377. 



Commissioner of Patents and Trademarks 
Washington, D.C. 20231 




Paul Bell 
Art unit 2675 
3 November 2003 




CHANH NGUYEN 
PfHMARYEXAa^iNER 



